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FOREWORD 



A special review learn was appointed by the Director of the 
National Science Foundation, H. Guy ford Stever, for the purpose 
of reviewing the NSF program in pre-college science education. 

The review team, as directed, completed a detailed study of 
pre-college science curriculum development and implementation 
activities. The development of policies and operational 
procedures, were traced over the past quarter of a century and 
their impacts on management practice were analyzed. A careful- 
ly delimited study of all curriculum development projects was 
carried out and five representative projects were examined in 
depth. Background studies by outside consultants were com- 
missioned and a substantial body of data was collected for use in 
preparation of the report. 

The report of tne review team is presented in two parts— Vol. 
I.— Findings and Recommendations and Vol II.— Appendix. Vol. 
I summarizes the extensive work of the review team. It presents 
the general findings of the study, along with the principal obser- 
vations of the team, and highlights the major policy issues that 
formed the basis of discussion for a special meeting of the Ad- 
visory Committee for Science Education. Recommendations of 
this Committee as submitted to the Director, NSF, are included. 
Finally, recommendations made by the chairman of the review 
team are presented for consideration by the Director and the 
National Science Board. 

As might be expected in any detailed review of a complex 
judgmental process, the review team identified many areas of ad- 
ministrative practice that could be improved. These include, but 
are not confined to, the need for bettei- definition of internal 
documentation, more explicit criteria for selecting reviewers, 
and clarification of procedure'followed in the modification of 
proposals in response to reviewer comments. While all are of im- 
portance to proper program performance, they can and should be 
addressed satisfactorily through normal administrative action, 
and, therefore, have not been taken up in detail in the report. 
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Vol. II— Appondix will include many of the basic documents 
used to arrive at the findings and recommendations of Vol. I. 
They include an historical overview of the factors impacting on 
NSF policies and programs in pre-college curriculum develop- 
ment and implemention, detailed summary case studies of five 
major projects, publication policy and financial arrangements, a 
summary audit and curriculum development financial 
arrangements. Attention is invited to the two contracted 
background papers which address important issues of 
curriculum implementation and commercial curriculum 
devolopm ^nl activity. 

In m^ opinion, the report is straightforward, factual and 
speaks directly to important issues. It is remarkably comprehen- 
sive considering the relatively short time available to the 
research team and supporting staff. Under the direction of Dr. 
Joel Snow, the team remained objective throughout and provided 
a balanced analysis that^honestly addresses both strengths and 
weaknesses of the program. 

I want to express my appreciation to the review team 
members and to the other members of the staff who contributed 
much time, thought, and effort to the development of this repjort, 
often with considerable personal sacrifice. 

Robert E. Hughes 

Assistant Director/National and International Programs 
Chairman, Science Curriculum Review Team 



EXECUTIVE SUMMARY 



On April 7, 1975 the Director, National Science Foundation appointed a 
review team to undertake a detailed study of the pre-college science 
curriculum activities of the NSF. The team has completed its study and has 
submilled its report in two volumes; Volume I, Findings and Recoiiimen- 
aalions. and Volume II, Appondix. 

After extensive investigation and development and analysis of a substan- 
tial body of datcT, the review team has made the following observations: 

• The program has been instrumental in bringing about a major change 
in the content'of science teaching materials at the pre-college levd 

• No comprehensive review of future needs for pre-college curricula 
has been carried out, although' there has been extensive change in the 
national situation with respect to these curricula 

• There is a need for substantial reexamination of the policy 
framework of this program 

• Project management decisions conformed adequately with policies, 
procedures, and practices in effect at the time decisions were made 

• General NSF management practices were consistent with policies in 
effect at the time 

• While there has been steady improvement in operational and 
management practices, additional improvements can be made. 

Five general policy issues were identified. These are: 

• Redefinition of the NSF role in curriculum development 

• Determination of future needs 

• Open competitive process for selection of awardees 

• Strengthened proposal review and project evaluation processes 

• Redefinition of the NSF role in curriculum implementation. 

These observations and issues were discussed in detail on May 9-10, 
1975, among members of the team and the Advisory Committee on Science 
Education. The Comniillse then made the following recommendations: 
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• NSF has a continuing rolo in science curriculum development at the 
pre-college level; NSF should not avoid controversy at the expense of 
educational and scientific value; 

• NSF should take an active and continuing role in determining needs 
for improvement of pre-college science education 

• NSF should use a broad range of granting mechanisms, including ex- 
panded use of **program solicitation" procedures 

• All large scale projects should have detailed evaluation plans and 
make provision for external summalive evaluation 

• Developers should be encouraged to make arrangements for publica- 
tion, manufacturing, and marketing without a requirement for NSF 
funds, 

Thechairman-of-the.re.vJew .leam,_after consideration of the results of the 
study and discussion with members of the NSB, made the following 
procedural and policy recommendations. 



PROCEDURAL 

• A needs assessment program should be initiated to develop and es- 
tablish priorities for curriculum development 

• Procedures should be developed to guarantee broad dissemination of 
needs, competitive proposal procedures, review by qualified 
professionals and, when appropriate, pilot testing of competing 
courses 

• Formal in-depth review of completed curriculum development 
programs should be carried out 

• More formal, structured procedures should be established for 
periodic review of ongoing major curriculum development programs 

• Barriers to broad diffusion of new curriculum materials should be^ 
researched and results broadly disseminated to allow and encourage 
Stale and local authorities to exercise their total responsibility For 
adoption of curriculum materials 

• The NSF should ensure, by legally binding agreements, that all 
curriculum development grants include the NSF disclaimer clause. 
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POLtCY 



The National Science Board should develop a definitive policy state- 
ment on the purposes and objectives of NSF curriculum development 
activities. This statement should delineate the extent to which future 
activities in both the natural and social sciences should be directed 
toward the objectives of supporting science training specifically for 
individuals embarking on science careers, and science education for 
all students to illuminate the underlying nature of our technological 
world. 

The National Science Board should formulate a clear policy state- 
ment on the role of the NSF in natural and social science curricula im- 
plementation. Since curriculum implementation activities are 
designed to disseminate materials that are sometimes regarded as 
controversial or political in nature, a clear policy is needed for the 
guidance of future activities. 

Mechanisms for administering curriculum implementation that 
allow NSF to remain at "arm's length" from the process are needed. 
New approaches should involve State and local authorities, private 
institutions and academies. 
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I. Purpose and Design of the Study 



Dr. H. Guy ford Stever, Director of the National Science Foundation es- 
tablished a specie.! Science Curriculum Review Team to undertake a detailed 
study of all pre-cullege science curriculum activities of the NSF. The review 
team was charged with developing an analysis of the policies, procedures, and 
practices of the program from its inception to the present. Further, the team 
was to determine whether NSF procedure was adequate to ensure that: 

• proposed subject matter fits within reasonable limits or norms with 
respect to educational value; 

• scientific content is accurate; 

• course developers are responsible and competent persons; 

• institutional and contractual arrangements are sound. 

Additionally, Dr. Stever stressed that: 

• the study and analysis should be in all respects independent and ob- 
jective, 

• the examination of cases and experience should be complete and un- 
biased by our previous practices, 

• scrutiny of the fiscal and management approach should be thorough 
and unhindered by past commitmenls, 

• potential or real conflicts of interest must be carefully addressed, 

• NSF policies and practices are to be v^arefuily scrutinized to ensure 
that the appropriate role of NSF in curriculum development is being 
followed. 

Dr. Stever further directed that the study team ensure that: 

• a thorough examination of past practices is undertaken, 

• a rigorous analysis of business and contractual relationships is 
developed, and 

• positive recommendations for improving the program s practices are 
developed, 

"A fully effective analysis of these issues is essential to honest 
examination of the integrity of our curriculum programs. 
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**As you know, I am commilled to reporting our conclusions lo 
the Congress after tippropriale discussion with the Advisory Com- 
mittee for Science Education and the National Science Board. Your 
work is a crucial element in. formulating this report. Let me urge you 
to require that in every respect that this study will be d model of ob- 
jective and professional analysis." 

In his letter of April 1, 1975, to Congressman Olin E. Teague, Chairman of 
the House Committee on Science and Technology, Dr. Stever outlined the 
structure of the analysis to be undertaken. 

")n,my NSF review I plan lo have investigated the pre-college 
curriculum development activities in a broad sense, including 
MACOS pariicularly and also the program more generally. To do 
this, some procedural questions will be studied; for example, the dis- 
tribution rights and royalty arrangements, In addition to a general 
survey of all of the curricula that have been developed, I shall have 
the review team make a detailed study of several cases as well as 
MACOS to see what they illustrate about the procedures that NSF 
has used in the support of curriculum development. The review will 
examine the following: 

(A) Curriculum Development Program 
- (1) History 

(2) Case Studies 

a. CHEM Study (Chemistry curriculum), 1959-1972 

b. Science Curriculum Improvement Study, (SCIS) 
1962-Present 

c. Man: A Course of Study (MACOS) 1963-1970 

d. Comparing Political Experiences (CPE) 
1972-Present 

e. Individualized Science Instructional System 
(ISIS) 1972-Present 

(B) NSF Distribution Policy and Royalty Arrangements 

(1) History 

(2) National Science Board Policy (1969) 

(C) Curriculum Implementation Procedures 

(1) History 

(2) Research Studies Regarding Implementation 

(3) Current Practices 

(D) Evaluation Procedures for Establishing Content and Utiliza- 
tion 
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(K) Pniclices niul Proccdurcis in Science Curriculum DGveiopcd 
by Other Organizations 

(F) Recomniendations 

Dosignatod members of the team included: 

Dr. Roborl E. Hughes, Assistant Director for National and International 

Programs (Chairman of the Review Team) 
Dr. Grover E. Murray. Member, National Science Board, and President, Texas 

Tech University and Texas Tech University School of Medicine 
Dr. L Donald Sh.elds, Member, National Science Board, and President, 

California State University at Fullerton 
Mr. Robert B. Boyden. Audit Officer 

Dr. Eloise E. Clark, Director, Division of Biological and Medical Sccnccs 
Dr. James D. Cowhig, Deputy Director for Public Sector Produclivilv 
Mr. Walton M. Hudson. Budget Officer 

Dr. J. Arthur Jones. Program Analyst, Office of Planning and Rf.30urc,;s 

Management 
Mrs. Maryann B. Lloyd, Deputy General Counsel 
Mr. Leonard A. Redecke, Contracts Administrator 

Dr. Joel A. Snow, Director. Office of Planning and Resources Management 
(Executive Secretary of the Review Team) 

Many other momhois of the NSF staff with scientific, grant and contract, 
financial managomont, and research management experience were called 
upon to contribute to the study. Members of the Education Directorate staff 
provided source material and assisted in the development of factual data. It is 
important to note that none of the members of the review team had worked in 
the programs being investigated. 

A major effort was undertaken to analyze practice and procedure in five 
curriculum development projects. The five examples were chosen to represent 
a wide range of fields of science and are at differing stages of program 
devolopment; one is totally completed and "closed out/' two have completed 
the development phase and are being implemented; and two are still under 
development. It is believed these examples provide a representative view of 
the evolution of NSF procedures for curriculum development and implemen- 
tation. 

Project case studies were carried out by review team task groups which 
also included experienced NSF officials with professional backgrounds in 
both science and business practice. Each task group was instructed to ex- 
amine the grant records over the lifetime of the project and to document how 
each significant project decision was made. 
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In the examination of project files the review and monitoring history of 
each project and the decision making process were analyzed. 

The results of this detailed analysis were transmitted in writing and with 
full documentation as internal working papers, to the Executive Secretary of 
the review team. After completion and critique of this analysis, each task 
group prepared a summary report describing the curriculum project and 
developed recommendations for possible improvement of program practice or 
policies. Appendix 4 comprises the summary case study reports. 

A general survey covering a sampling of all 53 NSF curriculum develop- 
ment projects was also undertaken to determine whether the issues raised in 
the course of the case studies were isolated instances or were more 
widespread. In addition to the above analyses, working documents were 
developed which related to each of the items identified in theoutline (page 2). 
Particular attention was paid to the business and financial elements of the 
projects. The distribution policy and fiscal history were reviewed and audit 
reports were prepared on each of the case study projects. A summary of these 
audit reports is found in Appendix 5; the individual audit reports are on file. 

The roles of NSB, the Congress and the Executive Office of the President 
in developing NSF policy in the pre-college curriculum area were examined. 
In addition, background studies were commissioned on the practices of the 
publishing industry and its relationship to government funded projects and 
on the extensive research literature relating to curriculum implementation. 
The Office of Education and the National Institutes of Education provided in- 
formation about their practices, and finally, an analysis of evaluation and 
oversight procedures was prepared. These working documents and the case 
studies prov ide the basis for the analysis presented in this report; many of the 
documents can be found as appendices to this report. Volume II. 

The final report was prepared by a group assembled by. and under the 
direction of the Executive Secretary of the review team. 



il. Findings, Observations, and Policy Issues 



This section contains a general survfty of curriculum projects and syn- 
opses of five case studies chosen for detailed analysis by the review team. 
Also presented are observations that represent a consensus of the review 
team. Policy issues that were identified during the study are organized to 
reflect the flow of activities i.; curriculum development projects and were dis- 
cussed jointly by the members of the review team and the Science Education 
Advisory Committee in meetings held May 9 and 10, 1975. 

Director Stever charged the review team and Committee with ensuring 
that: 

• the proposed subject matter fits within reasonable limits or norms 
with respect to educational value 

• the scientific concept is accurate 

• the course developers are responsible and competent persons 

• the institutional and contractual arrangements are sound. 

The list of 53 curriculum projects follows. More than half were reviewed 
in some depth by the team. Detailed case study summaries from which the 
synopses were drawn are to be found in Volume If, Appendix 4. 



LIST OF CURRICULUM PROJECTS 



Curncij/um Pro/ecf Current Status 

Physical Science Sludy Commillee (PSSC) Completed 

**2. School iVIalhemalics Sludy Group (SMSG) Completed 

3. Chemical Bond Approach (CBA) Completed 

**4. BSCS Biology Completed 

*r). CHl-M Sludy Completed 

6. Elcmunlary School Science Pro j eel — University of 

Ca'^'^">'">'> Completed 

7. Khimcntary School Science Project— University of 

Illinois (Atkinl Completed 

8. TV Pro«^run for Malheuuitics Teachers Completed 

9. Syracuse Webster Mathematics Projects (Madison) Completed 

**I0. Elementary Science Study (ESS) Completed 

Anthropology Curriculum Study Project (ACSP) Completed 

**I2. S(:i(inc(!-A Process Approach Completed 

la. University of lilinois Committee on School Mathematics Active 

14. MINNKMAST Terminated 
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*ir). Science CiMTiculum Improvement Study (SCIS) Completed 

*I6. Earth Science Curriculum Project (ESCP) Completed 

17. School Science Curriculum Project — University of 

Illinois (Sallinger) Terminated 

*I8. Man: A Course of Study (MACOS) Completed 

19. Elementary School Science Improvement Project— University 

of Utah (\Vood) Terminated 

20. Secondary School Science Project (Time. Space. Matter) Completed 

*2I. Introductory Physical Science (IPS and PS 11) Completed 

TL Films for In-Service Education of Teachers of Elementary 

School Mathematics , Completed 

23. Quantitative Approach in Elementary School Science Completed 

24. Hi«^h School Coui^se in Modern Coordinate Geometry Compleied 

25. High School Geography Project (HSGP) Completed 

26. Sociological Resources for the Secondary Schools (SRSS) ... Completed 

27. Engineering Concepts Curriculum Project-Man Made World... Completed 

28. Elementary Mathematics Projects (Arithmetic Project) Completed 

29. Harvard Project Physics (PP) Completed 

30. Portland Interdisciplinary Science Project Completed 

*3I. Intermediate Science Curriculum Study (ISCS) Completed 

32. Improvement Project in Mathematics for Subcultural 

Groups (Gibb) Completed 

33. Secondary School Mathematics Curriculum Improvement Study 

(SSMCIS) Completed 

34. Computer Based Self-Instructional Course for Supplementary 

Training of Secondary School Teachers of Physics Completed 

35. Environmental Studies for Urban Youth (ES) Completed 

*36. Comparing Political Experiences (CPE) Active 

37. Biomedical Interdisciplinary Curriculum Project (BICP) Active 

38. Demonstration and Experimentation in Computer Training and 

Use in Secondary Schools (Dartmouth) Completed 

39. Boston University Mathematics Program Active 

40. Development of Computer Simulation Material ^ 
(Huntington II) Completed 

41. Development of Teacher Training Materials in Math 

(HCTH) - Completed 

42. Experimental Teaching of Mathematics in Elementary School 

(Suppes) Terminated 

43. Exploring Human Nature (EDC) Active 

44. First Year Algebra with Application Project Active 

45. Human Behavior Curriculum Project (APA) Active 

46. Human Science Program (HSP) Active 

*47. In(lividualiy.ed Science Instructional System (ISIS) Active 

48. Mathematics Problem Solving Project Active 

49. Mathematical Resources Project (Hoffer) Active 

50. Outdoor Biology Instructional Strategies (OBIS) Active 

51. Project for the Mathematical Development of Children Active 

52. Technology-People-Environment (TPE) Active 

53. Unified Science and Mathematics for Elementary 

Schools (USMES) Active 



• 5 Caso StudiiNS 
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FINDINGS 



Findings of the review team members are summarized in the General Sur- 
vey of Curriculum Projects and synopses of the five case studies. 

11 must be strongly stressed that the material presented here is structured 
to identify and highlight problem areas and does not reflect the overwhelming 
extent to which the procedures and administrative activities represented 
good judgment and sound management. 



GENERAL SURVEY OF CURRICULUM PROJECTS 

To fulfill the Director's charge, the review team sought to identify any 
problems— past, present or future— of possible public concern. More thin 
half of the 53 projects were screened for indications of procedural ineffec- 
tiveness and to determine the degree of compliance with established 
policies. Six procedural areas were considered in the survey: (a) needs 
assessment, (b) proposal generation, (c) proposal evaluation, (d) develop- 
ment monitoring, (e] evalual ion of materials and processes, and (f) 
arrangements for production and distribution. 

Nine sample projects, in addition to the case studies mentioned, were 
selected for a relatively intensive review. They were chosen because they 
represent a significant fraction of the total NSF obligations in the field— an 
^'^ggregate obligation of $95 million— which, along with the $30 million 
obligated for the five case studies, covers 76 percent of the total amount 
obligated for curriculum development and implementation, 

NEEDS ASSESSMENT 

Strengthened external review procedures will be required if effective 
mechanisms are to be created for the determination of needs for new 
curricula. This is illustrated by some of the following problems identified in 
the course of the study. 

Typically, a needs assessment related to a particular field is performed 
by a conference of scholars and educators in that field. Experience has 
shown that it is commonly assumed by conferees that a need in /act exists, 
and that the question of how to improve a curriculum is to be addressed. 
This assumption of need during development of certain projects has 
resulted in a less than clear definition of goals. Lack of specificity in early 
development stages, reliance on conclusions of study groups on other topics, 
and the modeling of projects on existing examples has at times created 
doubts concerning the need for a new course. 

In some instances, projects have been spin-offs, initiated in fields not 
previously taught as separate disciplines; others have been initiated 
primarily because they complement existing projects; and yet others are 
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open to quesliun bucauso of their sheer number, e.g., twenty projects in the 
field of mathematics. Heavy expenditures for preparation of texts in one 
project drew criticism because it was argued that private enterprise was 
meeting the need. 

Procedures requiring a systematic and independent assessment of the 
need for projects are less than adequate. Clearer statements of needs would 
improve NSF's ability to generate a broader base of interest in potential 
development projects, provide a framework upon which competitive 
proposals could be evaluated, and could enhance the basis for monitoring 
development and evaluating results. 

PROPOSAL GENERATION 

Ordinarily a proposal to undertake curriculum development is sub- 
mitted in response to informal encouragement by the Foundation of a source 
whose credentials have been established and who is perceived as respon- 
ding to a potential need. In this survey no case was found where competitive 
proposals for the same project exist. Further, the Foundation has taken no 
explicit position on competitive solicitation. 

Criticism could be directed at certain proposjil generation practices 
which may have led to questionable results. Examples include the charge 
that formation of a corporation was a direct result of one project grant; that 
some proposals have been developed in direct consultation with the Foun- 
dation; that other projects have been transferred from one institution to 
another along with the project director; and that there have sometimes been 
no records to indicate how the transfer or acquisition of a particular project 
was effected by a grantee. 

A more normal competitive solicitation procedure, if instituted, could 
result in more focused delineation of specific needs, more definitive short- 
and long-term planning and should elicit more detailed, and in some in- 
stances, more realistic cost estimates. 

PROPOSAL EVALUATION 

The need for outside review prior to initial funding- becomes evident 
when it is realized that an aw^ard in seven figures for a continuation of work 
has occasionally been granted after review by one individual, albeit subject 
to approval by higher management. Initial funding, it appears, has in some 
instances been granted without benefit of more than cursory staff review. 

Proposals to initiate major phases of a project are ordinarily subject to 
mail and NSF divisional committee review. Mail reviewers' complaints that 
some proposals were too extensive or complex to be given more than a global 
appraisal are supported by many of the reviews examined by the team. 

Often, in current practice, evaluation is performed by a single project 
director w hich has led to his "owning" it and becoming the ultiniate authori- 
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ty on its nature, success and value. While this procedure is of merit in a 
research program, it is less felicitous in these costly projects that are design- 
ed to meet specific public needs, and which would benefit from expanded 
outside review. 

As in other procedural matters, judicious proposal evaluation would 
encourage development of explicitly stated goals, well formulated plans, 
and a realistic budget clearly related to the project plan. 

MONITORING OF DEVELOPMENT 

While the Foundation's monitorship of most projects appears to have 
•been broad, vigorous and responsible, it appears to have lacked a systematic 
approach. The files examined reveal extensive but occasionally incomplete 
records on administrative, financial, and technical matters, and it appears 
that most contacts with NSF staff were initiated by grantees. Few progress 
reports were found, and those on file were normally submitted with°a re- 
quest for renewal of support. 

Advisory groups are not required by NSF but are usually formed by 
grantees to monitor projects under development. If the Foundation relies on 
these groups, it runs the risk of becoming captive to the grantee whose 
creature the advisory group almost inevitably is. True neutrality in such an 
autonomous arrangement is difficult, and the deliberations of these groups 
would appeal- to merit more stringent monitoring. 

Generally, a policy of neutrality has been maintained on substantive 
matters, but where the Foundation has been the sole source of support for a 
large grantee organization, it has usually accepted responsibility for the 
grantee's salary and employment practices. In a few cases, this practice may 
have resulted in excessive disbursements or an inappropriate degree of 
Foundation intervention in fiscal practices of the grantee. 

There is further evidence that suggests the need for increased monitor- 
ing. Major awards have sometimes been deemed necessary to continue ef- 
forts on certain projects in order to glean useful materials from earlier dis- 
appointing performances. Lack of monitorship has resulted in continued 
support of some projects where the aims of the Foundation and the grantee 
have, over time, diverged; where tangential activities have been pursued to 
the detriment of the goal; and where empire building tendencies on the part 
of the grantee have become evident. Weaknesses could be corrected by more 
active monitoring and by clarification of policy on Foundation support of 
revision costs and by setting maximum personnel per capita costs. 

A model or detailed plan is needed to provide a framework for 
evaluating status information. Such a model should include a basis for 
evaluating the acceptability of subject matter, accuracy of its content, the 
competency and responsibility of potential course developers, and the 
soundness of institutional arrangements. 
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EVALUATION OF MATERIALS AND PROCESS 

The quostion here is to determine whether materials and processes un- 
der development respond to the needs, goals and specific objectives 
originally addressed in the project plan. This differs from the "formative" 
monitoring and evaluation described above. An effective example of this 
type of evaluation is demonstrated by Tho Nalional Longiludinal Sludy of 
Mulhanmlical Abilities in which students were foljowed for five years to 
determint effects of conventional, SMSG, and other new course sequences 
on /erformance in mathematics and science. 

'.jck of a systematic procedure for evaluating project results has left the 
Foundation vulnerable to such criticisms as sponsorship of materials 
development beyond its legitimate sphere, or support of courses which 
touch upon value-charged matters in a possibly insensitive or ineffective 
way. Materials cited have included those which apparently treat evolution 
as a fact rather than a theory; deal, however obliquely, with religious in- 
stitutions, ethnicity, human behavior, birth control, and reproduction; and 
those using living materials. It is not claimed that science related to these 
subjects should be excluded from support, but rather that NSF lacks a 
systematic procedure that would ensure monitoring of the use of potentially 
controversial materials and that would also ensure careful decision making. 

Elitism is yet another charge that has been leveled at some projects— 
those which lead to the benefit of but a few— while the content of others has 
been judged too difficult for the average student targeted. Following the 
adoption of some courses, the level of enrollments has actually dropped. 
Random reactions to implemented courses on the part of evaluators have 
run the gamut from "too highly structured," * of limited usefulness," "too 
V igorous," to the expression of belief that teachers are not always adequate- 
ly prepared to handle the new course material. Another reason gi\ en for lack 
of widespread adoption of some courses is simply that they cost too much. 

A more systematic procedure for the evaluation of projects after im- 
plementation should be viewed as a high priority need in science curriculum 
development. 

ARRANGEMENTS FOR PRODUCTION AND DISTRIBUTION 

Such arrangements are usually made by the developer, but are normally 
subject to Foundation approval of subcontracts and disposition of income. 
Some evidence of procedural inadequacy was found in this area where 
Foundation approval was either not sought, or informally given, perhaps 
because most action is between grantees and subcontractors. However, free 
and effective competition seems to have been a normal practice, with con 
tracts awarded to the lowest fully qualified bidders. 

Authors paid from grant funds were commonly permitted to publish 
their work commerically, provided their royalties were paid over to the 
grantee. 
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Records iiulicdto thdt income from development projects has been close- 
ly controlled by the Foundcition. Typically, it is deposited in a special ac- 
count and disbursed only with Foundation approval. Nearly $12 million 
have been rt^turned to the Foundation by sixteen of the projects. Revolving 
funds were set up to finance publication in a number of the projects sur« 
veyed. 

A recurrent issue appears to have been w^hetherincjme could be used to 
finance the revision of curriculum materials. It is clear from the records ex- 
amined that NSF has not adopted an unequivocal position on this question. 

Recognizing that the selection of subject matter is a responsibility of 
school authorities, the Foundation, has, as a rule, limited the use of grant 
funds to the development of new materials cmd the dissemination of infor- 
mation about them. It has left puhlicalion and distribution arrangements to 
private enterprise. While it relied on its grantees to manage these processes, 
the Foundation has maintained ultimate control over them and is therefore 
indirectly accountable for the widespread distribution of experimental text- 
books." Produced as paperbacks, these trial textbooks may have competed 
during the test phase with regular published editions. This suggests another 
area requiring policy definition by the Foundation. 



SYNOPSES OF CASE STUDIES 

The five case studies used were chosen for extensive examination 
because Ihej are at different stages in the total process, because they repre- 
sent different disciplines and reflect ways in which methods of development 
may have varied over the years. Again, the aim was to identify program 
procedures and potential problems, thus most findings relate to ad- 
ministrative practices rather than general policy. 

Administrative decisions are necessarily contingent upon a policy 
framework. The foregoing survey and the case studies, synopses of which 
follow, served as an aid in staff and Committee development of the Obser- 
vations and Policy Issues presented in this section. 

SYNOPSIS OF CHEM STUDY REVIEW 

The CHEM Study project which was initiated in 1960 and continued until 
1972, had as its goal "organization of a chemical educational materials study 
to prepare, through research and study, textual and experimental material to 
aid in development of a modern chemistry teaching program for U.S. high 
schools." 

The published material developed in the CHEM Study program includes a 
textbook, laboratory manual, teacher's guide, a series of achievement ex- 
aminations, other supplementary programs and 29 films.^ Additional reading 
lists, wall charts and lab equipment items were developed. 
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Dr. GlcMin T. Seaborg, University of California, Berkeley, was asked to 
assume leduerbhip for the proposed revisions. A steering committee of ex- 
perls v\as established to develop a detailed plan for content; compile lists of 
Lonlribulors, writing groups, and trial teachers; set target dates and develop 
an outline for initial testing in high schools. Following endorsement from N&F 
and after review by the NSB. an award to initiate the project was made. 

By 1965, in addition to its widespread use in the U.S., the material had 
been adopted in many parts of Canada, India, New Zealand and Australia, and 
by 1968 the material had been translated for use in 13 foreign countries. 

In the U.S. the materials were widely and quickly adopted and their use 
persists to the present, with minimal use estimated to be 25% of all high school 
students. CHEM Study materials, including revisions and derivations run 
well over 60% usage. 

Review and Oversight History 

These mechanisms reflect traditional practices of the Foundation. By 
definition this requires auquaintance and communication with scientists 
carrying out the work. 

The CHEM Study piup* sal received staff review and review by experts 
outside NSF prior to that of the NSB. The steering committee remained as the 
advisory and policy group and gave over^dl direction to the project. NSF 
i:loselj monitored developments and pcuiicipated in meetings with the Steer- 
ing Committee throughout the development, revision and evaluation stages 
and thus was able to represent its viewpoint. 

Monitoring History 

NSF staff followed the development of the project closely. Fiscal reports 
and annual reports of progress were required in the terms of the award. At a 
managerial level, the NSF staff was essentially in continuous contact with the 
progress of the project. Informal reports were frequent. advi(,e and confirma- 
tion of proposed directions were given prior to undertaking the activity. 
Questions in\ olv ing selection of publisheis and distributors, royalty income, 
etc., were foi warded to NSF legal counsel for verification of compliance with 
NSF and Federal policy. The pattern of advance planning with NSF feedback 
and subsequent action was firmly established in the CHEM Study project. 
The project directors, in turn, made conscientious efforts In keep NSF fully 
and currentlj apjM'ised of progress tmd solicited advice on anticipated direc- 
tions and problems. 

NSF staffmade frequent site visits to the central project office at Br keley 
and to regional centers after they were established. They attended planning 
and v\ riting sessions and meetings of the Steering Committee. A CHEM Study 
nev\ slet ter v\ as de\ eloped and formal descriptions of the jurse were publish- 
ed in a wide variety of educational journals. 
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In 1966 accounts were audited by NSF. and required minor chanoes con- 
cemmg mdirec. costs no improprieties in the use of funds we., oun 1 
first d^-^ft'o Th I" 1 was built into the project from the outset. The 

n m l 1 r P"' ^° '''' classrooms within three 

months. Reactions of teachers and students were fed back to the proiec"diror 
or after use of each section, thus, modification and revision of th tex w s " 
^o^rZoe' '-''^'^^^^^^ teachers were^H^ 
By the second year, the text was virtually complete. The teacher's onide 
was developed almost entirely from teachers' suggestions n su seou n 

S7e ^h^Mr' i^r 

back r'l "ivisiZ. '' ""^'^-^"'^'^ °^ '^^^'"S. evaluation, feed- 



Suggestions for revision, identified problem areas, and student and 
0 Cher evahuuions were assimilated and acted upon under the ac visem nt of 

ucK. available for testing by the end of the first summer's work. 

» 

Contractual Arrangements 

In December 1960 the CHEM Study staff solicited proposals from those 

ff nVr'""''?""''"° ^••^•-^^"^-8 the materials. After review 
llic staff and Steering Committee recommended W. H. Freeman and Co bo 

CH:r. fi n ^''^ ^'-^''^^^^^'^'-y - all i^spe ts Subse- 

quent film arrangements were equally satisfactory 

the cSl StulfvTnl''? 'V'^' Foundation's files, it appears that 

tne CHLM S udy project was managed in accordance with NSF's and NSB's 

.1.0 ^,sr"i;!lyr Appendix'"' a-ns-^nts is found in 

Implementation/Dissemination History 

It was determined that the materials produced should be comDetitivo 

sTl ct?:nUl!;:, io !'"^' r'^'''''' ^« ^'-^^'^^ in ordeTo a ; 

I Mmtori r h '"^^ 'f'^ adaptat on of both equipment and supplemen- 
m t OPS ^ '^'P^'"'^ r ''^-'y discussions at NSF, with advisory com- 

n 1 ees and project personnel recognized that "product acceotance" md 
utilization were the ultimate goal for this efforL^At th Tame time NSF 
repeatedly reiterated its position that its funds could not be use 1 frp^omo 
ion and distribution of the materials and that selection of materials or us in 
schoo s resided with State and local authorities. Funds could, however be 
used for the dissemination of information about the project, anSrc eed'th 
grantee was obligated to publish reports on the project in national jou 'als 
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Early in the study, inadoqualo loacher preparation in the sciences was 
rerooni'/ed to he as much a probhim as poor curricula. Gradually, ihesummer 
inslikiles adopted CHEM Study materials, and as teachers became ac- 
"u/ul^ed vvith 'them, materials were adopted for use in classrooms arounc the 
country. The newsletter and publications generated many ^'^YTnZ-nTw 
sullanl assistance in adopting the materials in schools as vvoH as Pnv Y 
spLored institutes. In addition, a series of conferences were m.t.atedfm^^^^^^ 

personnel. Except for this initiative, no other formal .mplementat.on 
mechanisms were considered for the CHEM Study Program. 



SYNOPSIS OF SCIENCE CURRICULUM IMPROVEMENT STUDY (SCIS) REVIEW . 

The SCIS project currently under way at the University of California at 
Berkeley, begun in 1962 and scheduled to continue until 1977. is focused on 
developing a framework of fundamental science concepts related to students 
own expo^-ience with natural phenon.ena. To date, SCIS has developed 
several ungraded, sequential physical and life science programs. 1 here are six 
units for a physical science sequence, and six for a li e science sequence lor 
el i ie. I ry grades and one unit designed especially for kindergarten Each 
has been cai^fully evaluated by SCIS staff during development and all were 
tested throughout the Nation prior to publication. , . , , , 
The six basic physical science units are designed to introduce and 
develop concepts leading to science literacy; the life sciencesequence focuses 
on organism environment interaction. Either may be used independently, but 
units within each are designed to be sequentially P''«sented 

Class materials are in the form of a kit for a teacher and 32 chi dren con- 
taining all materials save standard classroom supplies and certain Iresh 
water organisms sent separately on requesl Each complete kit cos s between 
$125 ancrS280; refills cost between S8.00 and S60.00. A complete set ol the K-6 
SCIS kits would cost approximately S2.700. 

The SCIS program is designed to foster laboratory-type experiences in 
which students deal directly with live animals and real objects. Objectives for 
each activity are well specified in the teacher's guide, but are not presented in 
student texts to avoid hindering the explorational and experimental aspects 

of g^jg programs effectively, the developers recommend that the 

teacher have sufficient background in science and Ihe program to prolit Irom 
its inherent flexibility. The developers have therefore maintained a strong 
emphasis on in-service training for teachers at SCIS headquarters in Berke ey 
and at the other SCIS trial centers. Other conferences and courses have been 
made available through the publishers, developers and other colleges and un- 

SCIS units and materials are presently being used to some ^xtent in 
almost all states. Sales estimates by the publisher (Rand McNally & Co.) m- 
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clicaU. thai men. than 3% or approximately l million students in the elomen- 

^^luUlT ^P'^^' '-"^ ^977 a minimum of 

o/o ol the school aood population will have boon exposed to the program The 
program has been modified for use by blind children 

.nnnrl' 7"""^"-'' f}^ d«P^'"^5 on whether school personnel are willing to 

o Inil?;'' " V""""^ f^'"^'^ °" elementary school science, and 
on (Mloctivo Irainino ol loachors. 

Review and Oversight History 

F.-om inception of the project through December 1970, formative evalua- 
sio nrl ' "f Evaluation generally moved from discus- 

r HH-; n ll^o 'Exploratory version to classroom trials, revision and 

or n 97 " l''")"' 1970. NSF supported an evaluation workshop, and in 
^ ^love opmcnt of evaluation supplements was begun o serve 

lo^^hers who needed external assistance to evaluate student performance 
heso were completed .n 1974. Pl.ns for a summative evaluation are bein o 
developed by NSF staff. ° 

Monitoring History 

minim.;i"!"l :f. ^""'^ "'^^ P"'^"""^' ^PP^^'^ ^'^^^^ 

n-on i : Q to stimuli 

1 on the SC S projec director. Major interactions between NSF and SCIS 
slall appear to have taken place only directly following NSF receipt of annual 
SCIS proposals or when SCIS personnel suggested budget changes. 

Contractual Arrangements 

NSF personnel approved the selection of all publishers/distributors for 
m r nil f/'l .';f'''''V.° f contractual arrangements between SCIS 

(U.C Berkeley) .nd the publishers/distributors. Sound legal and business 
practices appear to have been followed. business 

Dissemination/Implementation 

Characteristics of SCIS implementation followed the following pattern: 

1. School leaders obtain funds, arrange training. 

2. Pilot run of programs. 

3. Key people are chosen, attend program at Berkeley involving train- 
ing, observation, conferences. 

4. Key personnel train local teachers. 

...r\t P"^"^h«^ quarterly which reaches more than 25,000 

leaders. In the summer of 1974 NSF funded implementation projects 
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nalionvviile vvilh 1600 paiiicipanls involved extensively with SCIS materials, 
concopts, philosophy and teaching methods. 

The summary case study of the SCIS project is found in Appendix 4 and is 
accompanied by a list of perceived problems. 



SYNOPSIS OF MAN: A COURSE OF STUDY (MACOS) PROJECT REVIEW 

A3 described to the National Science Board on August 8, 1963, "This pro- 
ject is part of a comprehensive plan for developing a carefully integrated se- 
quential social science-humanities program for elementary and secondary 
schools/' A primary emphasis was on producing ethnographic film studies to 
deal with the questions; What is human about hui. an beings? How did wo gel 
that way? How can we be made more so? The curriculum was designed to in- 
troduce organizing ideas early and restate them frequently by the use of films 
of people in other, and apparently quite different cultures. Students were to be 
encouraged to learn to use anthropological and ethnographic methods and 
materials. 

During June 9-23, 1962, prior to any NSF support, Educational Services, 
Inc. (EsT) sponsored an Endicotl House Conference in Dedham, 
Massachusetts, ". . . to develop an overall unifying approach that would 
provide guidelines for structuring of a humanities and social studies 
curriculum running through the entire elementary and secondary sequence." 
Participants included some 61 persons representing social science disciplines 
plus those of law, history, physics, and education. 

A report on the conference, A Narrative Report 1962-1964 Social Studies 
Curriculum Program, November 1964 concluded that, ". . . the teaching of the 
general field of social studies and the humanities is desperately in need of 
Improvement in the elem9nt,.ry and secondary schools of this country." 

Proposals from Educational Services, Inc., and the American Council of 
Learned Societies (ACLS) received by the National Science Foundation on 
January 3, 1963, described a review of 250 existing social science films con- 
ducted in August-September 1962 by ESI and ACLS staff which resulted in 
criticisms regarding the minimal involvement of social scientists. New pro- 
jects were proposed which would deal with these shortcomings by having 
films made by social scientists and reviewed by experts; by developing films 
that could be used for children at all grade levels and would be edited for 
viewing by a general audience. The films would provide a flexible format and 
would be limited to that data which could bo best represented on film leaving 
much to the student to be interpreted, and which would also experiment with 

film innovations. , 
In addition, two conferences supported by the Office ol Education and 
NSF were sponsored by the President's Scientific Advisory Committee 
(PSACJ. Results reinforced the conclusion that there was an urgent need for 
course improvement in the social sciences and led to recommendations that 
the NSF support social science curriculum development projects. 



16 



Today. MACOS iiialorials arc (!Slimal(!ci to bt: in use in about 1,700 
schools in 47 stales. Estimates of students affected range from about 200^000 
to :{28,000. The materials are also being used in five Canadian provinces and 
in hngland, Scotland, North Ireland, and New Zealand. One report estimates 
that in 1970 the MACOS materials were being used by about 200,000 children 
and that the number of schools using the materials had increased from ap- 
proxnnately 375 in 1967 to nearly 1,700 in 1970. 

M-"erials on community response are mainly newspaper accounts, 
reports of very small-scale mail surveys, or anecdotal information. These 
materials do indicate that MACOS was a subject of conlrovei'sy as early as 
1971 and that NSF social science projects received cautionary Congressional 
comment in the mid-sixties. 

Review and Oversight History 

Prior to it.s support of MACOS-related projects, NSF supported develop- 
ment of secondary curricula in anlhropology. Each major MACOS-related 
award was subject to staff review and/or peer review, and was submitted, as 
revised on the basis of those reviews, to the National Science Board for ap- 
proval. It appears, then, that in the judgment of scientific peer reviewers, 
represonlalives of the educational community, staff of the Education Direc- 
torale. and Ihe National Science Board that the proposedsubjecl matter did fit 
wiliiin reasonable limits or norms with respect to educational value. Further. 
Ihereappeared to be no questionson accuracy of content nor doubt about com- 
polonce and experieince of the developers. 

The proposals that led to the development of the MACOS curriculum in 
1963 requested 8284,200 for the preparation of anthropological films. All four 
proposals were submitted to peer review. Awards were made for two pro- 
jecls; the other two proposals were withdrawn in August 1963. The awards 
did not involve either policy issues or levels of funds requiring NSB approval. 

On the advice of NSF staff, the same grantee organization submitted a 
consolidated proposal requesting 8618,315 for additional work on the two 
projects that had been supported and for incorporating work described in the 
two proposals that had been withdrawn. This consolidated proposal was nol 
submitted to peer review; NSF staff recommendations for support were based 
on results of evaluations of the separate proposals that had r'jceived peer 
review earlier. The staff recommendation for support, at a reduced biid<'el 
level, was submitted to and approved by the National Science Board. ° 

All subsequent proposals for continued substantive work received peer 
review and or review by NSF program staff. In each instance, the program 
staff summarized the major issues raised by reviewers and made recommen- 
dations for support to senior staff of the Education Directorate. In each in- 
stance, the recommendation was for a lower level of effort than had been 
I'Toposed and was submitted to and approved by the National Science Board. 
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Monitoring History 

The primary moans oi monitoring the MACOS projects appears to have 
been peer re\ iew and staff review of the proposals submitted for specific 
work elements. These reviews resulted in modifications in the budgets re- 
quested and occasionally resulted in modifications in the scope of work. 
There appear to have been only two site visits conducted during the course of 
the MACOS curriculum development. Major responsibility for the conduct of 
the work was with the principal investigators and a planning committee of 
social scientists and educators that was assembled by the grantee. 

Review suggests that monitoring of the project has been a comparatively 
weak point in management of the MACOS project. 

Contractual Arrangemer^ts 

Because of the unusual nature of the MACOS project and the teacher 
training requi- ements which Educational Development Center (EDC) insisted 
on being w ritlen into the contract, the lisk of locating a suitable publisher 
was vory difficult. The contractual arrangements between EDC and 
Curriculum Development Associates (CDA) which finally evolved, though 
somewhat at variance from the norm, appear to je fiscally sound and ade- 
quate for the purposes intended. 

Dissemination/Implementation Plan 

Major difficulties were encountered by EDC in developing and carrying 
out dissemination and implementation of MACOS curriculum. The 
publishers believed the program had four major liabilities that made 
marketing it a risk of capital, time, and personnel. 

• Unconventional concepts of the course; 

• The need for special teacher education; 

• The interrelatedness of media with function; 

• The cost of the program. 

In 1969 and 1970 (prior to EDC obtaining the services of a publisher) NSF 
supported a small number of regional centers strategically located in univer- 
sities and colleges in Florida, Orjgon, Colorado, Connecticut, New York, and 
other stales. The purpose of these centers w as to furnish information concer- 
ning the MACOS curriculum in the school districts in the region. When 
schools expressed the wish to utilize MACOS curriculum, appropriate 
teacher training was provided by the cqnters. These centers were supported 
by the Course Content Improvement Program for three years beginning in FY 
1969 and were discontinued shortly after CDA contracted to publish and dis- 
seminate MACOS curriculum. 
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Because of the innovative nature and unconventional concepts of the 
course. EDC felt strongly that teachers should receive specific training prior 
to touching MACOS in the school systems. As a result. EDC made this a re- 
quirement for publishing the course. CDA officials stated that they would "be 
insistent on a method and program of dissemination which provides the 
teachers who use the Coiirnu with full exposure to the use potentials conceiv- 
ed of by those who have developed it;" and would "seek a financial arrange- 
ment which pro\ides maximum opportunity for further development by the 
Coaler (EDC) of this and other teaching-learning courses/* 

CDA was given full responsibility by EDC to develop the production and 
dissemination program. The Price Information Sheet for materials developed 
for MACOS cont a ins the following statement: "All order'5 for classroom or 
film materials are subject to verification by CDA thai the purchaser has com- 
plied with teacher education requirement necessary for proper implementa- 
tion of the Course." NSF provided support for teacher training for MACOS 
through its Summer Institutes Program, its Course Content Improvement 
Pi *ogram, and its Cooperative College School Program. A multiplier effect is 
considered to be highly desirable b^ EDC. and trained personnel are expected 
to return to their school districts and train other teachers in the effective 
leaching of the MACOS course, as is the case with many implementation pro- 
jects supported by NSF. 

At the present time NSF does not fund any regional centers for dis- 
semination implementation and NSF staff have repeatedly asked EDC to 
delete reference to NSF support of "regional centers" from its publications. 

Detailed discussions of financial arrangements, implementation, and dif- 
fusion methods are found in the MACOS case study, Appendix 4, along wMth 
staff comments on procedures to date and perceived problem areas. 

SYNOPSIS OF COMPARING POLITICAL EXPERIENCES (CPE) REVIEW 

The American Political Science Association propo./;d to establish a 
Political Science Course Content Improvement Project fo/ elementary and 
secondary schools to design and develop new instruction materials for 
teaching government and politics in elementary and secondary schools. The 
Political Science Course Content Improvement Project was to consist of two 
components. One. the High School Political Science Curriculum Project, to 
develop instructional materials for use in senior high school civics and 
government courses; the other, the Elementary School Political Science 
Curriculum Study Project, has undertaken a study of political science educa- 
tion in elementcu*^ schools and. on the basis of this study, developed a set of 
guidelines for the procUu.tion of new instructional materials and media for 
grades kindergtu ten through six. They were designed to be used either in con- 
junction with existing curricula or as new programs in political science 
edu( tion. Hov\ever, in winter 1975, the decision was made not to support the 
elementary school project after the first year. 
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The high school project material was in tryout in 1974 and the second 
semester material is in preparation. The exact nature of publication is not yet 
determined. It may be one hard cover textbook or a series of separate units. 

Presently there are twenty-five pilot schools using the course material. 
There are also fifty affiliate schools which receive the materials but are only 
informally involved in pilot testing. 

Review and Oversight History 

The determination was made by NSF that political science is an ap- 
propriate subject material for the Foundation. Verification as to the accuracy 
of the scientific content was a responsibility of APSA in its activities in 
monitoring the curriculum dev elopment. At later stages, some portions of the 
material v\ere distributed to re\ievvers or summarized in renewal proposals 
and comment ;\as obtained regarding scientific content. Based on the com- 
ments of the re\ iewers, co-project directors were found to be highly qualified 
professionals and there were no questions as to their competency. 

Monitoring History 

No site visit was made prior to the award of the grant on March 13, 1972. 
A site visit was later made to Indiana University, asubcontractor to APSA. 

Possibly, because this project is in its early stages, there has been con- 
tinuitx on the part of the NSF program office personnel and reasonable atten- 
tion to monitoring of the program. Not unlike other grants, the major reviews 
occur annually when requests for additional funds are received. No specific 
oversight committee was appointed by the Foundation. The fact that the 
American Political Science Association was the named grantee, with respon- 
sibility for the overall coordination of the effort, appears to have led to the 
conclusion by NSF staff that the need for such a function was met. 

Contractual Arrangements 

Indiana University held a publishers conference in July of 1974 to present 
four Social Studies Development Center projects to potential publishers. The 
Uni\ ersit> was furnished with a copy of the Foundation's publication policy, 
v\ hich required approval of the plan to announce the availability of materials 
to all qualified distributors; selection of a particular publisher; and finally the 
proposed contract between the publisher and the grantee. A second con- 
ference is planned for fall 1975 to present more definitive material, and then to 
invite specific proposals. 

Dissemination/Implementation Plan 

The dissemination and implementation plan has not been fully developed 
at this point, but it appears that it will conform to NSF requirements. 
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SYNOPSIS OF INDIVIDUALIZED SCIENCE INSTRUCTIONAL SYSTEM (ISIS) REVIEW 

The ISIS projocl is focusod on developing a flexible, open-ended, inter- 
disciplinary curriculum thai will facilitate individualization of science in- 
struction at the high school level. 

ISIS will consist of approximately 80 short, essentially independent 
modules or minicourses, each requiring 2-3 weeks of classroom time. Each 
module will deal with a specific topic by presenting, in an interdisciplinary 
manner, the appropriate concepts from biology, chemistry and physics. Perti- 
nent information from the social sciences will be incorporated in units that 
deal with the social implications of science and technology. Mathematics will 
be included when necessary. For all modules, "excursions" will be developed 
which will permit the student to probe into some of the more complex aspects 
of the topic. In an effort to keep down the equipment costs for schools adop- 
ting ISIS, the laboratory work will make use of materials and apparatus nor- 
mally available in high school science labs. Guidance in implementing ISIS 
under a variety of situations will be provided by an instructional management 
scheme. Twenty-nine minicourses are now in the trial stage of development 
and are being field tested in several high schools throughout the United 
Stales. 

According to the project director the ISIS minicourses will not simply be 
a set of plans, but rather will be complete learning packages containing 
everything the teacher needs to provide students with individualized science 
instruction. The ISIS developers feel that such a complete package will free 
the teacher from much of the routine of a classroom and allow the necessary 
time for interaction with individuals or small groups. A few of the topics to be 
included are: buying and selling, people pressure, seeing colors, ways we 
learn, heart attack, house plants, home electrical appliances, fire and explo- 
sion, and energy supply and demand. Other courses are proposed and are be- 
ing readied. Two trial minicourses, "Human Reproduction" and "Birth and 
Growth" have in one case (Dallas, Texas) been refused by a trial center. 

ISIS trial materials are currently being used by more than 10,000 high 
school students attending the ISIS trial center schools located throughout the 
If.S. It is still too early to assess the actual impact of the ISIS program since no 
final versions of minicourses have been developed. According to the ISIS 
developers there is already a high demand for ISIS materials, and they plan to 
release minicourses as they are completed. 

Review and Oversight History 

In a grant dated June 23, 1971, NSF provided support for a conference held 
in October 1971 at Callaway Gardens, Georgia, which brought together 34 
experts to explore the feasibility of a new approach to high school science. 
The persons attending the conference represented a very wide range of ex- 
perience in science and school instructional materials development. Every 
branch of the teaching profession was represented including classroom 
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locichurs, scionce supervisors, school administrators, professors of science 
education and learning theorists. There were also representatives from 
various professional organizations. The conference participants cited what 
they felt was considerable evidence for the inability of the then current 
science programs to meet the needs of today's society, today's schools, and 
today's students. 

Criteria for reviewers included that they represent a cross-section of dis- 
ciplines, be prominent in their field, and required that they have experience in 
pre-college education. They were asked to base their evaluations on the merit 
of the idv^a, the feasibility and effectiveness of the procedures, the 
qualifications of project leaders and on budget factors. 

Two oversight committees were initially set up, but have been replaced 
by an advisory board that seldom meets as a unit, but in subgroups w^hich con- 
centrate on the areas of their expertise. Board members are listed in the case 
study, Appendix 4. 

The principal investigator has expressed the opinion that a summative 
evaluation could not be made for at least se\en years to determine what 
happens to chddren as a result of being exposed to a fully developed product. 
Further, criterion referenced testing is recommended rather than normative 
testing. The object is to determine how many children meet some specific 
criteria rather than to see if they achieve a given score on a test. Step-by-step 
details of the evaluation process are found in Appendix 4. 

In the course of the detailed examination of ISIS the internal documenta- 
tion leading to the award decision was reviewed. In particular, it was observ- 
ed that, as is often the case, the written peer reviews formed only a part of the 
process. After analysis of the reviews, the program staff made the essence of 
these rev iews available to the ISIS project team. The ISIS team then respond- 
ed at some length to the critical views raised by the reviewers. In addition, the 
ISIS staff met with NSF staff to further clarify various points. The program 
staff recommendations reflected these discussions in detail and indicated a 
staff decision that these interactions dealt effectively with the critical 
rev ievver comments. The recommendation for an initial trial phase with a sub- 
sequent decision-to-continue being contingent on an adequate indication of 
progress, is fully consistent with the review consensus. 

Awards to continue the project were based primarily on staff evaluation 
of progress which included site visits, review of progress reports, and other 
oversight procedures. 

Although the review and oversight is judged to be adequate, it is noted 
that the material provided to the NSB did not fully describe the interactions 
with ISIS staff which dealt with negative comments of reviewers. It was also 
obser\ ed that panel review of the initial and subsequent proposals might 
have had advantages over the mail review process. The observations are 
documented in the case study report. Appendix 4. 
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Monitoring History 

The Foundalion has monilorcd progress of the ISIS project through site 
visits, telephone conversations and correspondence. Results from a team of 
throe outside reviewers will be used by NSFslaff. Staff members have attend- 
ed meetings of the ISIS Advisory Board and have reviewed all recommen- 
dations and actions of the group. 

Contractual Arrangements 

The commercial version of the program will be published by Ginn and Co. 
who will retain exclusive rights until December 31, 1984, after which the ex- 
clusive publication rights expire. All arrangements are reported to be 
satisfactory. 

Dissemination/Implementation Plan 

Current plans call for release of completed minicourses in groups of 15 to 
20 over the next five years. The first lot of 10 will be released this year. With 
the release of subsequent minicourses, the possibilities for clustering them to 
build several varieties of courses for students of all abilities will increase and 
ultimately there will be many disciplinary as well as multidisciplinary course 
possibilities. By 1979 the ISIS developers expect to have published enough 
mmicourses to allow schools who wish to do so to totally replace their high 
school science programs with one of many locally determined alternatives. 



OBSERVATIONS 

These observations stem from consideration of materials prepared in the 
course of the study, and relate this information to our understanding of the 
practices in program management and program development in this an°d other 
Federal agencies. Practices of the school systems of the Nation and of the in- 
dustries that supply them with material were reviewed and views of the 
scholarly community were also noted. Specifically., the detailed case studies 
undertaken provided the opportunity to examine the nature and character of 
NSF program procedures as they have evolved and have been applied in the 
course of curriculum development activities. 

Observation 1: It is widely believed that this program has been in- 
strumental in bringing about a major change in the content of science teaching 
materials at the pre-college level. This has stimulated related creative efforts 
in development of teaching materials in other Federal agencies, in the in- 
dustry, and in the school systems themselves. 

Observation 2: There have been extensive changes in the national situa- 
tion with respect to pre-college curricula over the lifetime of this program 
which began, essentially, in 1958. Program guidelines and policy have been 
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rexised as iho climate changed and as NSF staff and the National Science 
Board became increasingly aware of the very complex issues associated with 
the dissemination and implementation of course materials. The policy 
framework has evolved in a manner that reflects changes in the external 
situation. However, no comprehensive review of future needs for pre-college 
curriculum seems to have been carried out in any breadth and depth. Current 
program emphasis is on completing and implementing the array of courses 
already under development; very little attention is being given to initiation of 
new courses during this present period of stocktaking. 

Observation 3: NSF program staff has conformed with acceptable and 
current procedures and policies since the inception of these programs. 
However, climate in the Federal Government with regard to program accoun- 
tability, openness and the involvement of outside affected parties implies a 
need to consider a substantial reexamination of the policy framework for this 
program and of the management procedures which are in use. 

Observation 4: Operational management decisions made by 'he NSF pre- 
college science educatio program staff reflected acceptable practices, 
procedures and policies thai existed at the time the decisions were made. Dur- 
ing examination of project files and documentation only minor uncertainties 
in project management decisions could be found, and these were few. 

Observation 5: General management practice-, as applied by NSF offices 
external to the pre-cpllege science education programs were essentially con- 
sistent with existing policy framework and procedures. There are, however, 
alternatives which could be considered to improve the effectiveness of NSF 
programs and which, in some cases, are contingent upon changes in policy. 
Other improvements can be effected by internal strengthening of current 
practices. 

Observation 6: The operational and management practices of the pre- 
coUege curriculum program reflect steady and consistent improvement over 
the past decade. Nevertheless, further improvements in program management 
practices are still desirable and should be considered. Some of these are: 

• strengthened monitoring of projects through site visits, telephone 
contact, and other techniques 

• improved attention to the determination and, definition of run-out 
costs 

• strengthened evaluation procedures 

• more extensive and detailed review of legal and business matters 

• more clearly established criteria for general program practice. 



24 



POLICY ISSUES 



Although the foregoing obsorvalions indicaled thai the management and 
decision processes in this program have been adequate in the context of ex- 
isting policy and practice, there are questions both of policy and of manage- 
ment practice that require increased attention. In retrospect, as with any 
judgmen al process, there were errors both of individual judgment and of ad- 
minislrdive practice of a typo which can be reduced by diligence but never 
eliminated. More serious are major policy issues which have been considered 
at limes in the past but which have not been fully resolved or which now re- 
quire re e x a m i n a t i o n . 

The review team has identified five general policy issues which should be 
addressed in determining the future course of this program. These are: 

1. Redefinition of the NSF role in science curriculum development 

2. Determination of future needs 

3. Awards process 

4. Proposal review 

5. Curriculum implementation 

These policy issues were identified in the course of the study, and reflect 
the flow of activities in curriculum development and implementation. They 
were presented to and discussed in detail with the Advisory Committee on 
Science Education. The Committee's recommendations along with those of the 
review team are found in Section III. 

Policy Issue 1. 

REDEFINITION OF THE NSF ROLE IN 
SCIENCE CURRICULUM DEVELOPMENT 

For both the natural sciences and social sciences, present and future 
goals and objectives require examination and a statement is needed 
to define more clearly whether the program should be directed 
toward education of future scientists, science education for all 
citizens, or both. 



BACKGROUND 

The pre-collego science curriculum developmenl and impleinenlation 
programs arose out of the perceived need in the late 1950 s for major improve- 
'riienl in the science content of public education. This was stimulated, in part, 
by the advent of Sputnik and also by the view of most scientists and educators 
that the scientific training of students entering college was weak and that the 
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content of pr(>-(.i)llege courses and texts was out of date. As a result, the initial 
courses supported by NSF were proposed by, and came to be directed by, 
scientists with extensive research experience and established scientific 
reputations but u ith relatively iittle experience in the elementary and set^on- 
dary classroom env ironnumt. This early trend led to some major successes but 
also created certain problems. Some of the early courses developed with NSF 
support were judged by schools to be too esoteric for a great number (jf 
students. Others were able to receive only limited distribution; local school 
sj stems w ere relut.tant to adopt them because of their novel features, difficult 
content and the need for extensive teacher training or retraining. 

Examination of early projects shows that review of the content and intent 
of the program was performed almost entirely by research scientists whose 
principcd concern was for the substantive content of the course rather than for 
its teachcd/ility, and who lacked familiarity with the milieu in which it would 
be used. By contrast, many of the reviewers of recent projects ai*e classroom 
oriented teaching professionals from local school systems and teacher's 
colleges. 

There seems to be adequate evidence that a major shift in pre-college 
science curricula has taken place. A key question, however, is the extent to 
which further Federal supp(jrt is, in fact, needed. Therefore, it is suggested 
thai a redefinition of NSF goals and objectives in pre-college programs is in 
order. 

One oi the crucial issues is whether projects should be oriented toward 
providing the base of experience in science methodology, attitudes and con- 
tent primarily for those students who will be the future scientists of the Na- 
tion, or whether they should be oriented towards science education for the 
general citizen. It must be noted that many individuals who will receive ad- 
vanced scientific training in college will not themselves become research 
scientists but will work in a wide variety of science-related jobs. It appears 
that the scientific pre-college needs of these students may not have been ade- 
quately met, and it is felt this need merits more extensive evaluation. Stilb 
these two groups are surely outnumbered by those individuals who will not 
study advanced science intensively or at nil. 

The question of who should receive intensive pre-college scientific train- 
ing is particularly germane in the natural sciences where an early start 
toward scientific creativity is often considered to be exceedingly important 
for future scientists. Conversely, such an intensive introduction to the sub- 
stance and method of science might not be appropriate for the individual who 
is unable to perceive a need for scientific education. 

In addition to the question of which students should receive such scien- 
tific training, course content must be considered carefully as well as the 
proper grade level for introduction. 

Development of social sciences curricula demands particular aw areness 
of the ways society views values that may be associated with the subject 
matter. This can substantially affect acceptance of courses, or at least public 
perception of them, 
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It is to bo hopod that a redefinition of the NSF role in these programs in- 
volvino participant M^presentative of all disciplines, from both academe and 
the classroom, will aid in determining what NSF should do as well as what it 
should not. 

Policy issue 2: 
DETERMINATION OF FUTURE NEEDS 

Establishment of broadly-based review groups would allow assess- 
ment of pre-college curricula through integrated studies; new starts 
could be deferred pending this review of needs. Othersystematic ap- 
proaches to needs assessment could be explored. 



BACKGROUND 

It would appear desirable to carry out broad based analyses of future 
needs for science curriculum development at the pre-college level. This could 
be done in a variety of ways; for example, contracted studies and surveys 
could be undertaken which would examine the attitudes of special groups- 
teachers, school administrators, scientists, and others. Alternatively, con- 
ferences or symposia could be planned for each discipline; or congressional 
hearings could be held where expert testimony is presented by all interested 
parties. Possibly NSF could charter a special commission to examine needs in 
science curriculum development, or the Advisory Committee for Science 
Education or other outside groups could be organized into task groups to 
specifically examine these needs. 

At this stage it is proposed that a very broad needs assessment be under- 
taken that would cover the spectrum of science disciplines for students of all 
ages at the pre-college level. Such an assessment could then be followed by in- 
dividual discipline assessments or other specific assessments after a need is 
identified in the broader scope. This would narrow the range of anticipated 
teacher-curriculum development projects to a relatively manageable number. 
The process could be designed to tap the experience of scientists, educators, 
and citizens otherwise involved in the educational process and would allow 
them to work toward a mutually agreed upon assessment of needs and re- 
quirements. 
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Policy Issue 3. 
AWARDS PROCESS 



Conlingenl on the syslemalic assessment of needs, an NSF program 
decision to proceed in a given area could be broadly disseminated to 
allow an open competitive process for selection of awardees as com- 
pared with the customary NSF practice of responding to unsolicited 
proposals. 

BACKGROUND 

The process by which NSF makes grants for curriculum development is 
patterned rather closely after the traditional NSF approach for research 
grants; however the tasks undertaken are very different. Research projects 
are tailored to suit the individual investigator and his innovative ideas, while 
a curriculum development project requires a team effort over a number of 
j^ears to ultimately produ^^e an educationally useful and commerically viable 
product. The present NSF approach of responding to unsolicited proposals 
from interested parties in the community may today be inappropriate. 

From another viewpoint, the use of the full machinery of Federal procure- 
ment contract regulations for the development of curricula might be equally 
as inappropriate. These procedures require arm's length competitive 
processes which would make it quite difficult for NSF staff or advisors to 
work with those submitting proposals to aid in defining and improving 
programs, and would limit the degree to which subsequent changes in the 
program, could be effected without exhaustive and detailed review. 

There are, however, intermediate procedures. For example, based upon a 
broad general needs assessment and subsequent specific assessments, the 
technique of program annoucement or program solicitation could be 
employed wherein areas of interest and general requirements for programs 
are identified, leaving proposing institutions or consortia free to propose or 
not propose after needs have been identified. Most often in the past, those who 
havo identified the needs through a conference or other process have become 
the agents in constructing a large proposal and there has been little opportuni- 
ty for direct competition. 

A phased, step-by-step process, in which two or three or more projects for 
program definition would be supported might yield a variety of different ap- 
proaches to a given problem. A restricted proposal competition could then 
follow in which the small number of groups who had defined the problem 
would be inv ited to develop a full-fledged proposal and would receivesupport 
for this purpose. From these, one a wardee might be chosen. This process could 
conceiv ably produce a more highly developed plan for each project; one which 
would result in an improved structure to permit evaluation and oversight. 
Similar techniques -have been used by other agencies of the Federal Govern- 
ment and by the RANN program at NSF. 



28 



Policy Issue 4. 
PROPOSAL REVIEW AND PROJECT EVALUATION 

Procedures for external, independent evaluation of project progress 
and content at all stages of development are important to ensure their 
accuracy and usefulness. These procedures could be strengthened 
and might include participation of practitioners and other "users." 
Such evaluations could take into account problems, needs and 
perceptions at the local level. 

BACKGROUND 

Evaluation of curriculum proposals has customarily taken place in a 
manner analogous to basic research proposals, that is, individual proposals 
aro usually sent to mail reviewers, and these and other reviewers are 
sometimes used as consultants on proposal site visits. Evaluation of the con- 
tent of ongoing projects is often carried out by an advisory board or steering 
conmittee that works with the director of the project. A major project may un- 
dergo three or more external peer evaluations in various stages of its funding 
over a period of seven or eight years. Curriculum development proposals 
might better lend themselves to panel review than to mail review- 
particularly if well defined review criteria are in use. However, this method 
might raise the question of cost effectiveness. 

There has been an increase in participation of practicing educational 
professionals in NSF evaluation processes, but by and large, the evaluation of 
conlcMit has been left to the project team. Although material is subject to peer 
review, the question is whether it might not be desirable to have an indepen- 
dent third party advisory board or panel check on the content and progress of 
major projects, particularly of courses I ha I may hav_G value-laden content. At 
the end of the development process, independent third party evaluation might 
be particularly useful. 

Policy Issue 5. 
CURRICULUM IMPLEMENTATION 

One appror-:h is for implementation to be supported only after an in- 
dependent, impartial external review has been undertaken covering 
need, content and potential benefit. If NSF is to continue to support 
implementation, the following elements are worthy of consideration: 

• well defined review mechanisms for judgment of content, 

• a more clearly defined process for dissemination/implemen- 
tation, 
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• more dofinilivc; policies on rights, royalties, exclusivity, and 
fiscal arrangements, 

• mechanisms for NSF revicnv of decision to proceed with im- 
plementation including periodic review by NSF. 



BACKGROUND 

An analytical study of research on curriculum implementation carried 
out as part of thisstudy' has shown that **thesuccessful implementation of in- 
no\ ations is genercdly considered to require a degree of change, capability and 
motivation not typically found in schools/' It indicates that "early and 
moaningful involvement of those who will implement change'' is essential, 
frecjucmt (,ommunication with the users of materials is necessary and that 
training or involvement of all levels of the user system, teachers, ad- 
ministrators tmd field of study specialists is essential. NSF has used many 
me(,hanisms to deal with these matters, including special institutes for ad- 
ministrators antl teachers and the regular program of teacher training in- 
stitutes. 

The issue of curri(,ulum implementation is not clearly faced in the present 
program, but is dealt with hoc when each curriculum approaches the im- 
plementation stage. It is customarily assumed that implementation is an es- 
sential go«d. However, there has not generally been a conscious, carefully 
designed evaluation of the product prior to the decision to proceed with im- 
plementation of the program. 

Because implementation involves precisely that phase of the program 
that might be labeled '^j^romotion" or "marketing", an explicit policy state- 
ment on the extent of implementation appears to be needed. It is probably 
desirable to have a much more carefully delineated stage of. first, the decision 
to implement and second, the plan or scheme by which implementation is 
carried out. 

One approach worthy of consideration would be to undertake implemen- 
tation or dissemination review jointly with the Office of Education and the 
National Institute of Education. It should be recognized that implementation 
has long been a matter of serious concern both to the NSF staff and to the 
Natioutd Science Board and that most of the possibilities that have appeared 
reasonable have long been subject to trial and error and have often met with 
success. 

Of particular importance is the refinement of business policies in order t(j 
ensure that NSF practice is fair and equitable, including development of 
greater consistency with practices in other Federal agencies. 
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III. Recommendations 



ADVISORY COMMITTEE FOR SCIENCE EDUCATION RECOMMENDATIONS 

On May 9-10. 1975 the Advisory Commillee for Science Education met for 
the prime purpose of reviewing the findings and observations of the science 
curriculum review 'earn. The Advisory Committee focused its efforts on the 
pohcy issues raised by the review team and its recommendations are struc- 
tured in response to those issues. The report of the Advisory Committee as 
made to the Director, NSF, is included in its entirety, 

ADVISORY COMMITTEE FOR SCIENCE EDUCATION REPORT 

At the re(|u(>sl of Dr. H. Guyford Stover. Director. National Science Foundation, 
the Advisory Comnuttee for Science l-ducation met May 9-10 to hear and consider 
the draft report of the NSF science curricuUim review team. This ad hocstudy team, 
headed by Assistant Director Robert t). Hughes, provided an extensive report to the 
Advisory Committee. In order to have an outside opinion, as recjuested by Dr, Ste ver. 
the Advisory Committee considered the five major policy issues raised in the report! 
The Advisory Committee has submitted the following recommendations to the 
Direclorv 

1. The NSF has a continuing role in science curriculum development in the 
mathematical, physical, medical, biological, engineering, social, and other 
sciences at the pre-college level, insofar as these subjects can be addressed ap- 
propriately at that level. In view of the NSF's contacts with the scientific com- 
munity in research and education, the Foundation can make a unique contribu- 
tion by bringing new. intellectually challenging science content, and teaching 
methods to elementary and secondary school students and their teachers. The 
program should be broadly aimed toward encouraging future scientists and 
technicians as well as preparing all citizens who will face increasing demands 
for informed decisionmaking with regard to the impact of science and technology 
on their li ves. In pursuing this role, the Foundation should be mindful of the fact 
that the final decision regarding the use of instructional materials and curricula 
is made locally. In other words, the schools must have genuine options from 
among which they m^»y select the approaches most suitable to address local 
needs and circumstances. 

It should be recognized that much educational innovation is. by its nature, 
controversial. It therefore follows from the Foundation's educational mandate 
that the Foundation cannot nnd should not attempt to avoid controversy at the 
expense of educational or scientific values in the creation or development of new 
materials. 

2. The NSF should take an active and continuing role in determining needs for im- 
provement of pre-college science education. The needs analysis will include 
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soliLilcilion uf buggebliuns from the suienlifit research community in traditional 
anil interdlMaplinarv arcMs, elementary and secondary school teachers, school 
administrators, science educators, students, and parents. While comprehensive 
needs analyses are in progiess, the Foundation should continue to consider 
proposals and make grant awards where the need has been clearly identified. 
The results of the needs analyses should receive wide distribution to the public 
and will provide a basis for future program development. 

3. The NSF should avail itself of a broad range of granting mechanisms. They 
should continue to accept unsolicited proposals and should expand the use of 
"program solicitation" procedures. Program announcements should specify the 
criteria for evaluation and selection. 

Curriculum development projects cannot always be judged by the usual 
"peer group" used in research project review. When appropriate, reviewers 
should include experts in the subject matter, in the process of teaching that 
material, in project evaluation, and in the administration and management of & 
such long term efforts in school systems. 

The NSF should be alert for unsolicited proposals that may be of high quali- 
ty, but lin;^ted applicability (e.g,, excellent curricula dealing with local 
ecological systems). The individual who works out such curricula may not be 
aware of NSF procedures and may need special help to become involved in the 
granting process. 

4. All large scale project proposals should include a detailed plan for evaluating the 
progress of the project at specified times during the course of the work. The NSF 
should provide for external summative evaluations of all major projects. 

Criteria of evaluation should include factual data, estimates of levels of per- 
formance and educational significance, cost effectiveness, and degree of nor. 
NSF support provided. 

One technique for external evaluation which should be considered by NSF is 
the use of third parties to engage in a debate of the merits and weaknesses of the 
project. These debates could be used to identify not only weaknesses of factual 
content, but possible questions of '"propriety of content" in the anticipated use of 
the material. This debate might be carried out before any widespread dissemina- 
tion of the curriculum is undertaken and a summary of the pros and cons might 
accompany the materials as they are disseminated. 

Students who have actually participated in these programs could be inter- 
viewed to include their reactions to the program, 

5. To assure availability to the public of Foundation supported curricula and 
teaching materials, the devMoper should be strongly encouraged to make 
arrangements for the pubLcation, manufacturing, and marketing of these 
curricula or materials without the requirement of Foundation funds for these 
programs. 

To improve teaching practice in science, the Foundation should support out- 
standing projects for teacher education and resource personnel development. 
These may concern the use of innovative curricula or material whose develop- 
ment was supported by the Foundation, but shall not be restricted to such 
materials. 
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CHAIRMAN, SCIENCE CURRICULUM REVIEW TEAM 
RECOMMENOATiONS 

The following recommendations wr.re submitted to the National Science 
Board and the Director by the chairman of the special review team Assistant 
Director. Robert E. Hughes. They represent the conclusions of the chairman 
after due consideration of the results of the study conducted by the review 
team. The chairman also consulted with two members of the National Science 
Board who were associated with the review team for oversight purposes and 
with the Advisory Committee for Science Education. 

Procedural Recommendations 

The following recommendations are procedural in nature. They are in- 
tended to strengthen ongoing practices and to develop a somewhat more for- 
mal, structured approach to curriculum development and curru...lum im- 
plementation activities of the National Science Foundation. They udii be im- 
plemented through normal administrative action, but this, of course, does not 
preclude the possibility that the National Science Board might wish to speak 
to these or related issues in a general policy statement. 

1. It is recommended that a continuing program be instituted to develop 
and establish priorities for curriculum development activities. 

This needs assessment program should utilize a variety of mechanisms 
such as panels, conferences, symposia, workshops, etc., to develop priorities 
within and among prospective disciplinary and interdisciplinary areas. 
These activities should include representatives with professional experience 
in education, curriculum development, and in the disciplinary fields of in- 
terest. Active participation should also be solicited from school ad- 
ministrators, teachers and the interested public. 

2. It is recommended that procedures be developed to guarantee: (a) that 
information is widely disseminated about needs for curriculum development 
activities that have been identified in certain specific or broad areas (b) that 
proposals received in response to such announcements are reviewed by ap- 
propriate panels in a competitive mode; (c) that the review panels are careful- 
ly constructed to provide a sound intellectual base and reasonably broad 
representation; (d) that, when appropriate, several of the best proposals be 
funded as pilot programs for approximately two years, after which time 
another competitive review should be conducted to determine whether one or 
more of these should be continued. 

Many innovative and important ideas and programs arise from un- 
solicited proposals from individual investigators. It is vitally important that 
neither the needs assessment program nor the competitive review procedures 
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be slruclured so as to discourage or preclude the submission of independent 
ideas to the National Science Foundation. 

3. It is recommended that more formal, structured procedures be es- 
tablished for periodic review of major curriculum development programs. 

These reviews are not to be confused with "formative evaluation" or 
"summalive evaluation" activities that are undertaken by the grantee as a 
normal part of the curriculum development activity itself. The reviews should 
be structured by NSF to include appropriate professional and public 
representation and should be directed toward an assessment of the merits of 
the total development activity. The review activity shouldinvolvean interac- 
tive panel and, whenever appropriate, an on-site visit with the grantee. 
Special care should be taken to ensure that the review panels, while free to ex- 
press their views, do not stifle freedom of intellectual inquiry and in- 
novativeness on the part of the investigators. 

4. It is recommended that a formal mechanism be established for conduc- 
ting an in-depth review of completed curriculum development programs. 

The review panels should be carefully structured to provide balanced 
professional oversight and appropriate public representation. Such a review 
should be mandatory before curriculum implementation activities are con- 
sidered. This should not preclude normal testing activities within selected 
school communities for the purpose of developing and revising elements of the 
curriculum programs. 

5. It is recommended that a research program be developed to carefully 
explore existing barriers to diffusion of new curriculum materials in science 
education.^ 

The program should be structured to provide basic information about the 
total system. The information should be made available to the Congress, 
Federal agencies. State and local authorities, the educational community, and 
the curriculum development community. 

Total responsibility for adoption of curriculum materials in school 
systems rests with local and State authorities. There appears to be 
widespread concern among many professional and public groups about the 
complicated process of introducing new curricula into local school systems. It 
is important that all interested parties fully understand every aspect of the 
curriculum implementation process in order to ensure that Federal activity in 
this area does not infringe upon local and State responsibilities. 

6. It is recommended that a legally binding codicil be added to every 
curriculum development grant to ensure that the grantees comply with the ex- 
isting provision that a disclaimer of NSF endorsement be prominently dis- 
played on alJ materials developed under the grant. 
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Allhough such a disclaimer is called for in the NSF Grants Adniinislra- 
lion Manual, it appears that the requirement has been ignored in some cases. 
This is an important issue and the disclaimer should bo made mandatory with 
legally binding agreements. 

Policy Recommendations 

1. It is recommended that the National Science Board develop a definitive 
policy statement that speaks to the purposes and theobjectives of curriculum 
development activity at the National Science Foundation. 

The fundamental objectives of curriculum development. activities at the 
NSF have gradually changed over the years. At first, the primary objective 
was to design curricula to attract the best minds and to improve the 
educational attainments of those students in secondary and primary school 
systems who were likely to embark upon careers as professional scientists. 
Gradually, another objective gained widespread acceptance. This involved a 
deliberate concern with providing a deeper understanding of the impact of 
science and scientific thinking on modern society to all students at every 
educational level. The first approach is directed toward developing com- 
petence in specific intellectual disciplines; thesecond toward illuminating the 
underlying nature of our technological world. The policy statement should 
delineate the extent to which future science curriculum activities at NSF 
should be directed toward those goals for both the natural and the social 
sciences. 

2. It is recommended that the National Science Board formulate a clear 
policy statement on the role of the National Science Foundation in the im- 
plementation of science curricula. 

In the public at large, in Congress and in the local and State educational 
communities there are widespread convictions that the role of the social 
sciences in school programs differs in some fundamental way from the role of 
the natural sciences. Traditionally, in many quarters, the social sciences have 
been regarded as a key element in introducing young students to the social and 
political environment in which they will live and work. Indeed, courses in 
local. S.tate and Federal government— civics, history, and geography— have 
often been directed to that end. From this point of view, attempts to introduce 
alternative approaches to the social studies are sometimes regarded as having 
political implications. 

Concern over new and controversial ideas is. of course, not limited to the 
social sciences; possible impacts of new developments in the physical and 
natural sciences have long been debated. In both cases, such concern is 
heightened whenever the Federal Government appears to be using its 
resources to deliberately disseminate or promulgate controversial material. 
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SiiK.u (;iini(,ulum implumunlaliun aclivilius are designed to disseminate 
nidlorialb that are sometimes regarded as controversial or even inherently 
political in nature, it is important that a clear NSF policy be developed For the 
guidance of future activities in this field. 

3. U is recommended that the National Science Board initiate a study to 
develop new mechanisms for administering curriculum implementation ac- 
tivities that will allow the National Science Foundation to remain at "arm's 
length" from the process. 

Analyses of the overall impact of NSF sponsored curriculum develop- 
ments in the scienc.es suggest that there has been widespread acceptance of 
many of the programs in the Nation's school systems. The national 
educational establishment is nut only huge, it is remarkably diverse and high- 
ly deccniralized. It is generally accepted that organized curriculum im- 
plementation activities are necessary if new developments are to be brought 
to the attention of local authorities for their consideration. Thought should be 
given to new approaches that might directly involve State and local 
authorities, private institutes or academies in curriculum implementation. 
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